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0.  Abstract 

Emerging  distributed  power- grid  systems  will  integrate  tightly-coupled  networks,  namely 
a  power  grid  and  a  communication  network,  to  provide  distributed  power  monitoring  and 
control.  While  distributing  power  monitoring  and  control  away  from  a  central  site 
enhances  the  robustness  of  the  power-grid  to  multiple,  dispersed  failures  and  attacks, 
such  communication-based  distributed  control  schemes  can  introduce  complex  cascading 
failure  scenarios  in  the  face  of  large-scale  WMD  attacks.  The  objective  of  this  project  is 
to  model  and  analyze  the  interactions  between  coupled  communication  networks  and 
power  distribution  grids  so  that  potential  cascading  failures  in  the  composite  complex  bi- 
infrastructure  network  can  be  examined,  analyzed  and  avoided.  This  fundamental 
research  work  is  motivated  by  fundamental  challenges  presented  by  potential  distributed 
power-grid  topologies  and  structures  currently  under  discussion  in  the  power  community. 
This  leads  to  a  framework  for  understanding  the  subtle  interactions  between  coupled 
power-grid  and  communication  networks  and  their  effects  on  cascading  failures.  Using  a 
combination  of  graph-theoretic  dynamical  modeling  of  cascading  failures,  optimal 
distributed  control-  algorithm  design,  and  distributed  estimation  techniques,  we  will  study 
how  various  communication  network-based  power  measurement  and  control  approaches 
impact  the  robustness  and  efficiency  of  power  networks  in  the  face  of  WMD-induced 
cascading  failures.  The  project  uses  this  abstraction  to  characterize  cascading  failures  in 
terms  of  system  parameters  such  as  power-line/substation  capacity,  communication 
connectivity  and  delay,  and  network/grid  topology,  as  well  as  parameters  related  to 
distributed  energy  resources  such  as  generation,  storage,  and  demand  response.  The 
proposed  technical  approach  will  then  provide  us  with  a  novel  design  framework  for 
increasing  adaptability  and  robustness  of  smart  power  grids  to  WMD-induced  cascading 
failures.  It  is  to  be  noted  that  funding  for  this  project  was  offered  only  for  the  first  year  of 
the  original  three-year  project. 

1.  Objectives 

The  objective  of  the  proposed  work  is  to  develop  a  mathematical  theory  to  analyze 
cascading  failures  in  coupled  distributed  power  grids  and  communication  networks 
resulting  from  WMD-induced  time/space  correlated  damage  to  the  power  and 
communication  infrastructure.  The  predictive  capability  of  the  mathematical  framework 
will  be  used  (1)  to  establish  an  understanding  of  the  vulnerabilities  associated  with 
cascading-failure  effects  in  distributed  approaches  for  power  delivery,  and  (2)  to  optimize 
the  control-agent  and  power  redistribution  design  as  well  as  power  grid  and 
communication  network  topology  to  minimize  cascading  failures. 
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2.  Specific  Tasks  according  to  the  Statement  of  Work 

This  effort  consisted  of  five  tasks  as  listed  below. 

Task  1.  Graph-theoretic  modeling  of  cascading  failures  in  distributed  power  grids  in  face 
of  time/space  correlated  WMD-induced  damage. 

Task  2.  Build  simulation  capabilities  by  porting  GridLAB-D  to  high-performance 
computational  platforms  and  coupling  it  with  the  OmNet++  network  simulator. 

Task  3.  Understand  the  logical  coupling  of  the  communication  network  and  the  power 
grid. 

Task  4.  Formulate  graph-optimization  problem  for  the  communication-network  topology 
for  robust  communication  under  WMD-induced  failures. 

Task  5.  Perform  simulation-based  analysis  of  smart  power  grids  for  system-parameter 
identification  and  optimization. 


3.  Status  of  effort 

All  tasks  have  been  completed.  The  project  resulted  in  three  IEEE  conference  papers  on 
power  systems.  These  papers  are  attached  to  this  report  in  the  Appendix. 


4.  Accomplishments/New  Findings 

Here  we  present  the  scientific  and  technical  accomplishments  categorized  according  to 
the  tasks  listed  above. 

4.1.  Graph-theoretic  modeling  of  cascading  failures  in  distributed  power  grids  in 
face  of  time/space  correlated  WMD-induced  damage. 

4.1.1  Graph-theoretic  abstraction  of  power  grids 

As  a  starting  point,  we  identified  key  power  grid  components,  parameters  and  their 
dynamics.  We  abstracted  the  power  grid  as  a  graph 
consisting  of  nodes  (generators  and  substations) 
and  transmission  lines  connecting  the  nodes.  We 
considered  the  power  flow  distribution  in  this 
graph  based  on  the  physics  electricity  law. 

However,  the  complexity  of  keeping  track  of  the 
detailed  power  grid  state  and  its  power  flow 
dynamics  in  an  analytical  model  is  0(2"),  where  n 
is  the  number  of  transmission  lines  in  the  power 
system.  This  rendered  our  initial  abstraction  non- 
scalable  to  large  grids.  To  alleviate  this  problem, 
we  used  the  novel  idea  of  equivalence  classes  in 
order  to  reduce  the  complexity.  To  do  so,  we  first 
identified,  through  simulations,  certain  power  grid 
parameters  which  play  a  critical  role  in  the 
dynamics  of  load  redistribution  and  cascading 
failures.  (We  had  built  our  simulation  capabilities 


Fig.  1.  Grid  configuration,  grid  state  and  the 
transition  between  the  grid  states. 
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using  MATPower  [4],  a  package  of  MATLAB  m- files  as  well  as  GridLab-D  for  the 
distribution  level.  More  details  on  the  simulators  are  given  later  in  this  report,  see  Section 
4.2.)  Based  on  our  simulation  results,  we  discovered  that  the  number  of  failed 
transmission  lines,  maximum  capacity  of  the  failed  lines  and  the  grid- loading  ratio 
(which  is  defined  as  the  ratio  of  the  total  demand  over  the  total  generation  capacity  of  the 
grid)  affect,  to  a  large  degree,  cascading  failures  in  power  grids.  The  use  of  number  of 
failed  lines  and  the  maximum  capacity  of  the  failed  lines  as  a  descriptor  of  the  “state”  of 
the  grid  enabled  us  to  partition  the  collection  of  all  power  grid  configurations  into  a 
collection  of  equivalence  classes.  Therefore,  this  partitioning  of  the  entire  collection  of 
grid  configurations,  as  depicted  in  Fig.  1,  reduced  the  complexity  of  tracking  the 
dynamics  of  the  power  grid  to  0(n).  In  Fig.  1,  each  large  circle  represents  a  grid  state  and 
the  topological  graphs  with  working  and  failed  lines  inside  the  circles  representing 
different  grid  configurations  with  the  same  number  of  failures  and  the  same  maximum 
capacity  of  the  failed  lines. 

4.1.2  A  Probabilistic  model  for  the  dynamics  of  cascading  failures  and  blackouts  in 
power  grids 

With  the  simplified  abstraction  of  the  power  grid  at  hand,  we  developed  a  novel 
probabilistic  approach  based  on  stochastic-regeneration  theory  to  model  the  dynamics  of 
cascading  failures  in  power  grids.  The  model  characterizes  the  probability  of  reaching  a 
blackout  of  an  arbitrary  size  in  any  time  interval  starting  from  an  initial  grid  state  with 
random  initial  failures.  By  using  this  model,  we  can  derive  the  distribution  of  the 
blackout  size  in  power  grids.  This  model  utilizes  the  power  flow  distribution  dynamics, 
graph-theoretic  abstraction  and  our  defined  equivalence  classes  based  on  the  key  power 
grid  parameters  affecting  the  cascading  failures.  In  the  following  we  present  some  of  the 
equations  and  the  theorem  we  developed  for  this  model. 

We  use  the  notation  B(cmax  ,/■  )(t,  M )  to  represent  the  probability  of  reaching  blackout  size 
M  or  larger  (a  blackout  with  M  or  more  line  failures)  in  a  time  interval  1  and  initial  grid 
state  S  =  (Cmax  ,  F)  and  initial  grid-loading  ratio,  Rd/g  •  Here,  Craax  and  F  are  the 
maximum  capacity  of  the  failed  lines  and  the  number  of  the  failed  lines  in  the  grid, 
respectively. 

Note  that  when  F  >  M  then  B(cmax  ,f  ft,  M  )  =  1.  Now,  we  consider  the  case  F  <  M  and 
derive  integral  equations  describing  the  probability.  By  exploiting  the  properties  of 
conditional  expectations  and  renewals,  we  can  write 

t 

0 

where  fx  ( s )  is  the  probability  density  function  (pdf)  of  x  which  is  the  time  of  the  first 
event  occurring  following  an  initial  condition  at  (e.g.,  the  first  line  failure).  Note  that 
B(Cmax  M  \x  =  s)  is  the  conditional  probability  of  reaching  a  blackout  with  at  least  M 
failures  in  the  time  interval  t  given  that  x  is  equal  to  s  (recall  that  the  first  event  occurs  at 
5).  We  write  this  probability  as 

FAt,M  I  r  =  s)  =  P{t  =  W  I  r  =  s}B*f'G  FAt,M  I  r  =  s,t  =  W)  + 

v'-'max’r/  v^max’rz 

^P{r  =  Ui  \  r  =  s}B*f'G  F)(t,M  \  r  =  s,t  =  U), 

ie^L 
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where  £/,  and  W  are  different  events  corresponding  to  the  failure  of  a  line  with  larger 
capacity  than  Cmax  and  failure  of  a  line  with  the  same  capacity  as  Craax,  respectively. 
Now,  we  state  the  following  theorem  on  the  probability  of  a  blackout  of  size  M. 

Theorem-The  probability  of  reaching  a  blackout  of  size  M  or  larger  in  a  time  interval  t 
and  from  an  initial  state  S  =  (Cmax  ,  F  ),  where  F  <  M  ,  and  initial  grid-loading  ratio,  Rd/g, 
is  characterized  by  the  following  differential  equation: 

clB*°'G  fM,M) 

-J£=g - =  -X(S,RDm)B^F)(t,M)  + 

/eL 

A  key  feature  of  this  model  is  that  the  transition  rates  between  the  states  of  the  model 
(which  represents  the  rate  of  dynamics  of  the  cascading  failures)  depend  on  the  state  of 
the  system.  This  model  is  among  the  very  few  models  which  represent  the  evolution  of 
the  cascading  failure  probability  in  time.  This  means  the  results  of  the  analytical  model 
shows  how  fast  the  probability  of  cascading  failure  increases  from  an  initial  state  in  the 
absence  of  efficient  corrective  actions.  The  parameters  of  this  model  are  estimated  based 
on  the  Monte  Carlo  power-system  simulations.  The  transition  probability  and  the  stability 
probability  of  the  power  system  states  are  presented  as  functions  of  capacity  of  the  failed 
lines  and  the  number  of  failed  lines  in  Fig.  2  and  Fig.  3,  respectively.  Furthermore,  this 
model  enabled  us  to  consider  the  effect  of  multiple  correlated  failures  (WMD-induced 
damage)  through  the  states  with  multiple  failures  as  the  initial  state  of  the  power  system. 


max 


Fig.2.  The  transition  probability  trom  three  initial 
states  with  different  maximum  capacity  of  the  failed 
lines  to  different  possible  states  S'  with  C'max  >  Cmax. 


0  50  100  150  200  250  300  350  400 


Number  of  line  failures  in  the  state  of  the  power  grid 

number  of  failures  in  the  state. 


Our  paper  on  this  model  which  was  submitted  to  the  IEEE  PES  General  Meeting  2012 
has  been  accepted  [1], 


4.1.3  Verification  of  the  analytical  results  with  power  grid  simulations 

We  used  our  power-system  simulation  capabilities  to  simulate  the  cascading  failure 
behavior  in  power  grids  such  as  IEEE  118  and  IEEE  300  test  cases.  The  observed 
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behavior  by  the  simulation  of  these  two  systems  was  similar.  As  an  example,  the  effects 
of  the  capacity  of  the  initial  transmission  line  failures  and  the  number  of  line  failures  on 
stability  of  the  power  grid  were  similar  in  these  two  test  cases.  We  further  applied  the 
regeneration-theory  model  to  power  grid  IEEE  118.  The  results  of  the  analytical  model 
are  in  agreement  with  Monte  Carlo  simulations  performed  with  the  IEEE  118-bus  system 
(please  see  Fig.  4  for  a  representative  result).  Thus,  we  can  conclude  that  the  model  is 
successful  in  capturing  and  predicting  the  power-grid  cascading  failure.  In  particular,  the 
analytical  model  enabled  us  to  predict  the  blackout  probability  for  different  initial 
failure(s)  and  with  different  system  parameters.  A  representative  set  of  graphs  showing 
the  capability  of  the  model  to  predict  cascading  behavior  is  shown  in  Fig.  5  below. 


Fig.  4.  Prob.  of  having  a  blackout  of  size  M,  as 
a  function  of  M,  for  IEEE  118-bus  system. 
Solid  and  dashed  lines  represent  calculated 
results  for  our  model  and  dots  and  crosses 
represent  Monte  Carlo  simulation  values. 
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Fig.  5.  Calculated  values  of  the  probability  of  reaching  blackouts 
with  M  =  40  or  more  in  IEEE  118-bus  system  in  20  min.  interval. 
Each  curve  is  parameterized  by  the  max.  capacity  of  failed  lines, 
(a)  Initial  state  has  5  failures  and  RD/G  =  50%,  (b)  initial  state  has 
10  failures  and  RD/G  =  50%,  (c)  initial  state  has  5  failures  and  RD/G 
=  90%,  and  (d)  initial  state  has  10  initial  failures  and  RD/G  =  90%. 


The  results  in  Fig.  5  show  quantitatively  that  in  cases  where  the  maximum  capacity  of  the 
failed  lines  is  large  (>  400MW  ),  the  probabilities  of  the  blackouts  are  larger  than  those 
corresponding  to  other  cases  where  the  maximum  capacity  is  low.  The  same  effect  can  be 
seen  when  the  number  of  initial  failures  is  large.  As  expected,  in  cases  where  the  grid¬ 
loading  ratio  is  large  the  probabilities  of  blackouts  are  also  large  compared  to  those  with 
smaller  values  of  the  grid-loading  ratio.  Moreover,  the  probability  increases  much  faster 
in  time  than  other  initial  grid  states.  These  plots  represent  evolution  of  the  risk  of 
blackout  in  time  and  how  quickly  the  risk  increases  with  time  if  no  action  is  taken  as  time 
passes.  The  results  are  consistent  with  the  physical  concepts  used  in  the  model  and  also 
simulation  results. 

4.2.  Building  simulation  capabilities  by  porting  GridLAB-D  to  high-performance 
computational  platforms  and  coupling  it  with  a  network  simulator 

4.2.1  Transmission  level  simulations 

We  used  MATPower,  a  package  of  MATLAB  m-files,  to  simulate  the  power  system 
transmission  layer.  The  transmission  layer  is  of  particular  interest  to  this  project  as 
cascading  failures  are  attributes  of  this  layer.  MATPower  helped  us  solve  optimal  power- 
flow  problems  and  simulate  the  power  flow  distribution  over  the  grid.  We  performed 
several  simulations  on  IEEE  118  and  IEEE  300  test  cases  with  MATPower  in  order  to  see 
the  effect  of  various  parameters  affecting  cascading  failure,  specifically  those  pertaining 
to  overload  conditions.  As  a  real  power  system  example,  we  decided  to  use  the  ERCOT 
(Texas)  power  transmission  network  because  it  is  large  enough  to  exhibit  cascading 
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effects  and  it  is  largely  self-contained  (with  only  a  few  connections  to  external  network), 
yet  it  is  treatable  for  our  studies.  We  extracted  relevant  data  from  the  ERCOT 
transmission  network:  from  GIS  to  a  format  that  is  compatible  with  MATPower. 
However,  there  was  insufficient  information  in  the  data  available  to  us  on  the  Texas  grid. 
To  remedy  this  deficiency,  we  used  our  understanding  of  the  topology,  load  substations, 
and  generators,  to  generate  good-faith  estimates  of  the  missing  information,  which 
enabled  us  to  successfully  simulate  the  power  flow  over  our  augmented  version  of  the 
Texas  power  grid.  We  also  simulated  some  scenarios  of  cascading  failure  over  the  Texas 
power  grid. 

4.2.2  Distribution  level  simulations 

The  GridLAB-D  source  code  was  compiled  on  the  high-performance  machine  Pequena 
(available  to  this  project  through  collaboration  with  Dr.  Susan  Atalas,  Director  of  UNM’s 
Center  for  Advanced  Research  Computing,  CARC),  an  SGI-Ultrix  platform  with  2 
TFLOPS  of  peak  processing  power.  We  worked  with  developers  at  PNNL  (regional 
power  company)  to  resolve  several  issues  related  to  running  Gridlab-D  under  Linux  in 
general  or  64-bit  Linux.  We  built  models  of  real-life  distribution  grids  from  data  provided 
to  us  by  our  local  utility  company,  PNM.  We  also  built  models  for  two  distribution  grids, 
each  with  approximately  2000  loads.  We  completed  the  model  of  a  real  distribution 
feeder  in  Albuquerque,  NM,  which  will  allow  us  to  determine  realistic  load-shedding 
scenarios  that  can  be  achieved  using  smart  grid  controls. 

We  considered  loads  that  can  be  controlled  by  adjusting  the  power  consumption  at  the 
“end  user”  level  (residential  and  commercial)  via  “smart  grid”  controls.  We  investigated 
by  means  of  GridLAB-D  how  much  control  can  be  exerted  at  the  end  user  level  under 
various  scenarios,  ranging  from  the  status  quo  to  a  futuristic  distribution  grid  with 
“smart”  load,  including  PHEV,  appliances  that  can  respond  to  price  signals,  and 
“community”  storage  in  the  form  of  large  batteries.  Our  models  enabled  us  to  indicate 
how  much  load  reduction  can  be  achieved  at  substation  nodes,  how  long  the  load 
reduction  can  last,  and  how  fast  it  can  happen.  We  used  the  results  from  the  distribution 
grid  simulations  to  determine  the  overall  time  and  magnitude  response  characteristics  of  a 
substation  (a  class  of  nodes  in  our  transmission  networks)  to  a  load  shed  request.  We  also 
used  MATPower  to  implement  certain  load-shedding  operations.  Load  shedding  is 
implemented  by  the  optimization  on  the  generation  and  load  distribution  over  the  power 
grid.  Another  significant  accomplishment  is  that  we  were  able  to  model  the 
TRAMWAY  11  distribution  feeder  in  both  GridLAB-D  and  OpenDSS,  performing  a 
complete  simulation  of  daily  operation,  including  a  number  of  distributed  (residential)  PV 
generators.  This  enabled  us  to  simulate  realistic  load  shedding  events,  as  described  by 
statistical  distributions  of  load  shedding  capacity  of  distributed  resources,  which 
collectively  provide  a  response  to  a  load  shedding  request  at  the  transmission  level.  Our 
load  control  model  and  the  results  on  their  effects  on  the  power  grid  are  published  in  [2]. 

4.2.3  Simulating  coupling  between  communication  network  and  power  system 
simulations 

We  studied  various  parameters  related  to  the  communication  network  that  may  affect  the 
cascading  failures  in  its  coupled  power  grid.  Example  of  such  parameters  are  the  average 
network  latency,  bandwidth  limitations,  and  queuing  of  messages  during  high  levels  of 


9 


congestion.  We  included  communications  module  components  in  our  GridLab-D  model 
and  test  their  capacity  to  perform  load-shedding  operations.  We  also  performed  parameter 
tests  to  investigate  the  effect  of  latency  due  to  faults  in  the  network  as  a  consequence  of 
attacks.  We  specified  a  simple  set  of  data  that  needs  to  be  exchanged  between  the 
communication  network  simulator  and  MATPower/GridLab-D  to  provide  the  necessary 
communication  latency  and  connectivity  information  to  evaluate  various  control 
strategies.  In  our  studies  we  adopted  the  realistic  assumption  designed  control  signals  are 
transferred  over  the  communication  network  upon  a  communication  request.  At  that 
point,  communication  network  simulator  simulates  the  outcome  of  the  communication 
request.  This  reset  operating  conditions  (via  the  input  file)  for  MATPower/GridLAB-D, 
until  the  next  communication  request,  and  so  on.  We  also  improved  our  understanding  of 
the  IEEE  test  grids  by  including  topological  information  about  these  grids  in  our 
communication  network  simulations,  which  previously  only  included  node  connectivity 
and  physical  distance  between  nodes.  This  allowed  us  to  analyze  the  geographical 
distributions  of  generators  and  loads,  allowing  us  to  locate  control  hubs  in  a  practically 
reasonable  way. 


4.3.  Understand  the  logical  coupling  of  the  communication  network  and  the  power 
grid 

4.3.1  Control  and  communication  system  structure  design 

We  considered  the  control  and  communication  system  of  the  power  grid  as  a  hierarchical 
system.  Control  and  communication  systems  of  the  transmission  network  and  the 
distribution  network  of  the  power  grid  are  two  levels  of  this  hierarchy.  Since  large 
blackouts  and  cascading  failures  are  attributes  of  the  transmission  network  in  power 
grids,  we  studied  the  cascading  behaviors  in  this  level  as  an  important  step  in 
understanding  cascading  phenomenon.  We  assumed  that  the  control  over  the  distribution 
network,  placed  below  the  substations  of  the  transmission  network,  would  perform  the 
local  control  actions  as  well  as  implementing  necessary  remote  control  signals.  We 
decomposed  the  transmission  grid  into  sub-systems  based  on  the  control  regions  defined 
over  it.  We  assumed  that  in  each  sub-system,  the  control  regions  utilized  controlling  and 
monitoring  agents.  These  agents  were  located  on  top  of  the  communication  nodes  and 
operate  and  communicate  over  the  communication  system  spanned  over  the  control 
region.  Examples  of  controlling  agents  are  control  relays  and  remote  terminal  units 
(RTUs),  which  performed  local  control/automation  and  execute  remote  control  signals 
from  system  control  center  (SCC).  The  SCC  is  the  critical  control  center,  which  makes 
region-wide  decisions  and  may  have  backup  centers  to  solve  the  single  point  of  failure 
problem.  However,  at  each  time  only  one  SCC  is  the  responsible  control  center  for  the 
region.  Where  to  locate  the  SCC  in  the  control/communication  system  of  a  power  grid  is 
an  important  decision  in  designing  reliable  control/communication  systems.  Hence,  we 
utilized  the  clustering  algorithms  to  identify  the  best  places  (according  to  required  power 
grid  and  communication  characteristics  we  defined)  to  locate  the  SCC  and  its  backup 
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systems  over  the  power  grid.  Examples  of  the  metrics  we  used  in  the  clustering  algorithm 
are  power  volume  passing  through  different  points  of  the  grid  and  the  reachability, 
connectivity  and  geographical  proximity  to  the  grid  components.  To  reiterate,  the 
communication  network  includes  two  categories  of  heterogonous  nodes.  The  first 
category  contains  monitoring  nodes  that  collect  and  relay  grid-information.  The  second 
category  contains  only  a  few  decision/control  nodes  that  are  directly  associated  with  the 
high-level  control  centers  described  above 

Note  that  there  may  not  be  necessarily  a  direct  (one-to-one)  correspondence  between  the 
nodes  in  the  communication  network  and  the  power  grid  components.  For  example,  a 
communication  node  may  be  associated  to  a  subset  of  power-grid  components  or  there 
may  be  auxiliary  nodes  covering  a  subset  of  components.  This  gave  us  flexibility  in  the 
design  of  a  robust  communication  network  to  improve  reliability  and  robustness.  We 
tried  communication  networks  with  different  characteristics  such  as  degree  of 
connectivity. 


4.3.2  Logical  coupling  of  communication  network  and  the  power  grid 

We  studied  the  cascading  phenomenon  in  the  power  grids  under  the  scenarios,  which  the 
initial  disturbances  in  the  power  grid  are  accompanied  by  other  system  vulnerabilities 
such  as  failures  of  the  communication  and  control  systems  that  transmit  and  implement 
critical  control  signals.  In  this  study,  the  vulnerabilities  in  the  control  and  communication 
system  were  coupled  with  the  load  shedding  mechanism  and  their  effects  on  power-grid 
cascading  behavior  were  revealed  by  means  of  simulation.  Load  shedding  is  a  critical 
control  action  when  the  system  must  be  reconfigured  to  accommodate  the  disturbances  on 
the  power  grid.  We  modeled  the  vulnerabilities  in  the  control/communication  systems 
(communication  system  failures,  control  system  failures,  and  physical  and  policy-based 
limitations)  by  formulating  an  optimization  problem  for  the  load-shedding  control  with 
new  constraints  to  capture  these  vulnerabilities.  This  approach  relies  on  the  direct 
modeling  of  control/communication  system  vulnerabilities  within  the  load-shedding 
formulation.  It  is  a  simple  yet  effective  approach  for  coupling  the  power  grid  with  the 
control/communication  systems  without  dealing  with  the  complexities  associated  with 
integrating  communication  system  and  power-grid  simulators.  We  studied  various 
scenarios  of  failures  in  the  control  and  communication  network  and  their  effect  on  the 
cascading  failures  in  the  power  grid.  We  also  considered  various  scenarios  with  different 
topological  location  of  the  failures  in  the  communication  network.  The  results  of  these 
studies  have  been  accepted  to  publish  in  IEEE  PES  General  Meeting  2012,  [3].  The 
results  on  the  effect  of  losing  load  shedding  control  on  the  cascading  failure  are  presented 
in  Fig. 6.  According  to  the  results  presented  in  Fig.  6,  for  a  fixed  amount  of  uncontrollable 
loads  over  the  grid,  the  scenario  for  which  we  totally  lose  control  over  certain  load  buses 
results  in  a  more  severe  cascading  effect  compared  to  that  for  the  scenario  where  we  lose 
control  over  a  portion  of  the  loads  in  the  load  buses.  Furthermore,  in  the  presence  of 
control/communication  vulnerabilities,  lines  have  increased  chance  of  becoming 
overloaded  and  hence  fail,  which  causes  the  load  buses  to  be  disconnected  from  the  grid. 
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Therefore,  the  total  amount  of  unserved  loads  in  these  cases  are  larger  than  that  in  the 
case  when  controlled  load  reduction  (necessary  load  shedding)  is  performed. 

Figure  7  represents  the  effects  of  different  topological  distribution  of  uncontrollable  loads 
over  the  power  grid.  In  Fig.7  we  consider  scenarios  with  five  load  buses  with 
uncontrollable  loads  randomly  selected  over  the  grid.  In  each  sub  figure  of  Fig.  7  we  have 
shows  the  results  on  the  number  of  failures  due  to  cascading  phenomenon  for  each  of  the 
10,000  realizations  of  the  randomly  selected  load  buses  with  uncontrollable  loads.  Note 
that  each  point  in  each  of  the  plots  corresponds  to  one  realization  (from  all  the 
combinations  of  5  buses  chosen  from  all  the  load  buses  in  the  grid)  of  the  five  randomly 
selected  load  buses  with  uncontrollable  loads.  The  results  in  Fig.7  show  that  both  the 
topological  location  of  the  transmission  line  failures  and  the  topological  location  of  the 
failures  in  the  control/communication  system  affect  the  cascading  behavior  of  the  power 
grid.  In  Fig.7  we  have  also  looked  at  the  effects  of  spatial  inhibition  and  clustering 
among  the  uncontrollable  load  buses.  These  effects  can  be  important  due  to  the  nature  of 
certain  disaster  events  that  may  affect  the  power  grid  and  their  control/communication 
systems.  (In  Fig.7,  the  square  markers  correspond  to  scenarios  with  inhibition  and  circles 
correspond  to  the  scenarios  with  clustering.)  We  observe  that  when  there  is  clustering 
effect  in  the  distribution  of  the  uncontrollable  load  buses  the  occurrence  of  cascading 
failures  are  less  likely  compared  to  the  case  in  which  there  is  inhibition  effect.  This  may 
be  attributed  to  the  ability  of  the  power  grid  to  isolate  the  problem  locally  in  the  case 
when  the  uncontrollable  buses  are  within  close  proximity  of  one  another,  which  may 
impede  the  propagation  of  subsequent  failures  through  the  grid. 

Probability  of  cascading  failure  as  a  function  of  the  minimum  total  capacity  of  the 
connected  lines  for  three  different  number  of  load  buses  with  uncontrollable  loads  over 
the  grid  is  shown  in  Fig.  8.  When  the  minimum  total  capacity  of  the  lines  connected  to 
load  buses  with  uncontrollable  loads  is  above  a  threshold  then  the  probability  of  having 
cascading  failures  drops  sharply.  This  phenomenon  is  shown  in  Fig.  8,  where  we  show 
the  probability  of  having  ten  or  more  failures  beyond  the  initial  three  failures  for  three 
values  for  the  number  of  load  buses  with  uncontrollable  loads.  This  can  be  explained  as 
follows.  Since  it  is  assumed  that  the  loads  on  the  buses  with  uncontrollable  loads  are 
fixed  if  they  have  lines  with  lower  capacities,  there  is  a  higher  probability  of  overloading 
such  line  in  the  case  of  contingencies.  Moreover,  it  can  also  be  seen  from  Fig.  8  that  the 
probability  of  having  cascading  failures  increases  with  the  number  of  uncontrollable  load 
buses. 
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(a) 

Fig.  6.  Average  (a)  number  of  failed  lines,  and  (b)total  unserved  load,  due  to  cascading 
failures  as  a  function  of  the  ratio  of  total  controllable  loads  over  the  total  load  in  the  grid. 


Total  uncontrollable  load  (MW)  Total  uncontrollable  load  (MW) 

Fig.7.  Cascading  behavior  of  the  grid  for  distributions  of  load  buses  with  uncontrollable 
loads  over  the  grid  for  two  scenarios  of  initial  failures 
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Minimum  total  capacity  of  connected  lines 

Fig.  8.  Probability  of  cascading  failure  as  a  function  of  the  minimum  total  capacity  of  the 
connected  lines  for  three  different  number  of  load  buses  with  uncontrollable  loads  over 

the  grid. 

4.4.  Formulation  of  graph-optimization  problem  for  the  communication-network 
topology  for  robust  communication  under  WMD -induced  failures 

The  topology  of  the  communication  network,  comprising  all  the  nodes  and  the 
connecting  communication  lines,  designed  to  maximize  reachability  between  the 
decision/control  nodes  and  the  monitoring  nodes  especially  in  the  case  of  component 
failure  in  the  communication  network.  The  design  of  the  communication  network 
structure  and  topology  includes  discovering  the  number  of  required  nodes  and  links  and 
their  topological  placement,  as  well  as  resilient  communication  protocols.  We  designed 
optimized  communication  networks  overlaying  the  IEEE-118  power  grid.  Using  the 
network  connectivity  as  a  metric  as  well  as  the  clustering  algorithm  developed  in  the 
previous  task,  optimization  was  performed  to  identify  the  best  five  locations  (nodes)  for 
locating  the  SCCs  described  above.  This  enabled  us  to  create  scenarios  for  robust 
communication,  with  the  most  robust  location  used  as  the  main  SCC  and  lower-rank, 
backup  SCCs  used  thereafter  in  the  event  of  disruption  in  the  communication  due  to  a 
WMD  attack.  Random  and  correlated  node-failure  patterns  were  induced  in  the  network 
simulations  as  the  optimization  was  performed.  In  the  communication  network  design  we 
used  a  number  of  topologies,  including  random  topologies,  full  mesh  and  scale-free 
topologies  with  various  average  nodes  degrees  (number  of  edges  at  a  node).  We  observed 
that  power  grids  with  coupled  communication  systems,  which  have  higher  connectivity 
probability  in  the  case  of  failures,  are  more  robust  in  the  case  of  contingencies.  In 
addition,  we  observed  that  the  topological  location  and  characteristics  of  the  failures 
affect  the  cascading  behaviors  in  the  power  grid.  The  reachability  information  extracted 
from  these  studies  can  be  fed  into  the  power-grid  analytical  model  and  the  power 
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simulators.  The  optimal  approach  for  such  data  collection  graphs  is  a  Steiner  tree,  and  the 
construction  of  optimal  Steiner  trees  is  known  to  be  computationally  intractable.  As  a 
result,  both  offline  and  online  mechanisms  for  constructing  robust  communication 
topologies  that  approximate  optimal  graphs  are  needed.  We  worked  on  optimizing  data 
aggregation  and  transport  in  unreliable  networks  by  determining  appropriate  gradient- 
based  routing  functions  to  optimize  data  delivery  reliability  and  efficiency,  and 
comparing  the  resulting  routing  structures  at  small  scales  with  offline-computed  optimal 
routing  trees.  We  studied  the  effect  of  structured  and  unstructured  communication 
networks  on  the  robustness  of  smart  grids  in  the  face  of  WMD  attacks. 

4.5  Perform  simulation-based  analysis  of  smart  power  grids  for  system-parameter 
identification  and  optimization 

With  increasing  penetration  of  renewable  power  generation,  primarily  from  centralized 
wind  farms  (transmission  level)  or  from  PV  (primarily  at  the  distribution  level),  there  is  a 
need  to  increase  the  capacity  of  demand  response  (DR)  programs  to  absorb  intermittency 
from  these  resources.  The  implementation  of  these  programs,  with  associated  control 
strategies,  is  synergistic  with  the  requirements  of  securing  the  transmission  grid  from 
WMD  attacks. 

Our  investigation  of  cascading  failure  mechanisms  has  shown  that  load  shedding  can  be 
an  effective  method  of  reducing  the  likelihood  of  cascading  failure,  if  implemented  in  a 
timely  manner.  However,  currently  load  shedding  are  generally  implemented  by  an 
operator  manually,  following  a  request  from  a  system  operator.  Moreover,  requests  are 
subject  to  human  judgment,  and  sometimes  fail  to  be  implemented  correctly,  in  a  timely 
manner  and/or  in  their  entirety.  An  automated  response,  making  use  of  energy  storage, 
would  have  two  benefits:  first,  its  implementation  would  not  depend  on  human 
judgement.  Second,  its  impact  on  end  users  would  be  less  severe.  For  example,  storage 
units  (thermal  storage,  community  electric  storage,  utility-scale  electric  storage)  may  be 
deployed  to  produce  a  certain  level  of  load  shed,  rather  than  resorting  to  rolling  black¬ 
outs  or  brown-outs. 

There  are  a  number  of  challenges  in  the  implementation  of  automated,  fast  acting  load 
shedding  strategies  for  WMD  protection.  First,  the  mechanism  must  be  compatible  with 
“normal”  grid  operation  mechanisms.  Second,  the  mechanism  must  be  compatible  with 
local  regulations.  For  example,  control  by  real-time  pricing  (RTP)  may  be  possible  in 
some  regions,  but  not  others.  Where  RTP  is  not  available,  other  control  mechanisms  must 
be  adopted.  Finally,  the  control  strategies  must  be  robust  -  for  example,  high-bandwidth 
communications  requirement  may  pose  problems,  especially  because,  in  case  of  WMD 
attack,  communications  networks  may  become  congested  or  even  inoprable. 

In  this  work,  we  made  the  assumption  that  thermal  storage  devices  were  installed  in 
either  50%  or  100%  of  commercial  buildings.  This  is  a  cost-effective  strategy  for 
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controlling  the  load  profile  on  the  network,  and  represents  the  largest  individual 
component  of  the  peak  load.  However,  similar  considerations  would  apply  to  electrical 
storage,  which  is  in  fact  subject  to  less  stringent  requirements,  since  power  flow  is 
bidirectional.  We  considered  two  strategies  for  load  control  -  real-time  price  and 
stochastic. 

With  real-time  price  control,  the  cost  of  power  is  a  function  of  the  residual  capacity  of  the 
system.  This  is  in  line  with  industry  practices,  and  is  also  compatible  with  WMD 
response,  since  a  WMD  attack  would  automatically  result  in  reduced  capacity.  The 
customers  use  automated  response  to  activate  their  storage  devices.  Thermal  load  is  either 
met  by  depleting  storage  or  by  purchasing  power  from  the  network,  depending  on  an 
internal  power  price  threshold.  The  threshold  for  meeting  thermal  load,  in  turn,  is  set 
based  on  the  state  of  charge  of  the  storage  -  a  high  threshold  when  state  of  charge  is  low, 
and  vice  versa.  The  threshold  for  charging  is  set  similarly. 

Stochastic  control  (SC)  could  be  used  in  regions  where  RTP  is  not  allowed  by  regulation. 
In  this  case,  the  utility  or  some  control  agency  would  send  a  signal  to  all  customers, 
between  0  and  unity,  representing  the  probability  of  activating  a  distributed  resource.  In 
turn,  each  resource  generates  a  random  number,  which  is  compared  with  the  utility  signal. 
Based  on  the  result  of  the  comparison,  the  device  is  either  activated  or  not.  With 
stochastic  control,  loads  can  choose  to  op-in,  and  be  financially  compensated  as  a  result. 

Our  results  have  shown  that  both  RTP  and  SC  are  effective  means  of  controlling  the  total 
load  on  the  grid,  and  can  respond  effectively  to  both  renewable  energy  intermittency  and 
load  shedding  requests.  Moreover,  the  bandwidth  for  communication  is  very  low  -  only 
requiring  a  common  signal  to  be  broadcast  at  regular  intervals,  typically  on  the  order  of 
minutes.  We  also  note  that  these  same  mechanisms  could  be  used  for  enhanced  readiness 
to  WMD  threats.  For  example,  it  is  possible  to  increase  the  storage  SOC  in  cases  of 
enhanced  threat,  simply  by  modifying  the  RTP  or  the  stochastic  signal.  In  other  words, 
the  operating  mode  of  the  network  could  be  modified  by  using  “normal”  control 
procedures,  in  response  to  the  risk  of  cascading  failure  identified  using  the  methodology 
described  in  other  component  of  this  report. 

We  also  performed  simulations  to  reveal  the  effects  of  the  control/communication  system 
parameters  on  the  cascading  failures  in  power  grids.  Our  simulations  have  shown  that 
vulnerabilities  in  the  control/communication  systems  can  increase  the  probability  of  large 
cascading  failures  initiated  by  small  disturbances  over  the  power  grid.  In  addition,  we 
observed  that  the  topological  location  and  characteristics  of  the  failures  affect  the 
cascading  behaviors  in  the  power  grid.  Since  inhibition  and  clustering  can  be  important 
attributes  of  WMD  attacks,  which  exhibit  great  level  of  spatial  correlation,  we  looked  at 
the  effects  of  spatial  inhibition  and  clustering  among  the  uncontrollable  load  buses.  These 
effects  can  be  important  due  to  the  nature  of  certain  disaster  events  that  may  affect  the 
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power  grid  and  their  control/communication  systems.  We  observed  that  cascading 
failures  are  more  likely  in  scenarios  where  there  is  inhibition  (negative  spatial 
correlation)  in  the  loads  for  which  we  lose  control  over  than  that  for  scenarios  where 
there  is  a  clustering  effect  (positive  spatial  correlation)  in  these  loads. 
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